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The Minnesota Department of Education (MDE) and the Regional Training Projects (RIPs) sponsor three 
training sessions per year for schools that are participating in Minnesota’s 2-year training sequence for Positive 
Behavioral Interventions and Supports (PBIS). This Spring 2016 school status and training report provides a 
summary of the current implementation status of schools in training and results from the Spring 2016 training 
surveys. This report is intended for all stakeholders to better understand the status of PBIS of schools in training 
and help to refine training and data reporting. More information about PBIS in Minnesota can be found at: 


www.pbismn.org. 


Wilder Research is contracted to evaluate the statewide PBIS initiative. As a part of the PBIS evaluation, surveys 
are conducted with school teams who are participating in the training sequence (one survey was completed per 
school team). The results of these surveys, school team attendance records from the RIPs, data system information 
gathered from the RIPs and from school teams, and data entered for the TIC (Cohort 10 only), TFI (Cohort 11 
only), and SAS (both Cohort 10 and 11) are the basis of this report. This report summarizes the results of the 
Spring 2016 trainings and PBIS implementation as of May 2016: 

=" There are a total of 52 Cohort 10 schools that are in their second year of training. 

=" There are a total of 56 Cohort 11 schools are in their first year of training. 


Training attendance 


Schools in their first year of training (Cohort 11 in 2015-16) receive two days of training at each of three points 
during the year, and schools in their second year of training (Cohort 10 in 2015-16) receive one day at each of 
three points during the year. 


Overall, attendance for the Spring 2016 trainings was very high. All school teams in Cohort 10 and Cohort 11 
attended spring training this year. Most team administrators from both cohorts also attended training; however, there 
were a few administrators from the Metro and North regions in Cohort 11 that did not attend training. Nearly half of 
administrators from the North region in Cohort 10 did not attend training. See Figure 1. 
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1. Spring 2016 training attendance by Cohort (number of school teams / administrators attended) 


Cohort 10 

Metro North South 
team - full 21 18 13 
team - partial 0 0 0 
team - not attended 0 0 0 
administrator - full 20 11 13 
administrator - partial 0 0 0 
administrator - not attended 1 7 0 
Total number of teams 21 18 13 

Cohort 11, day 1 Cohort 11, day 2 

Metro North South Metro North South 
team - full 19 18 19 team — full 19 18 19 
team - partial 0 0 0 team — partial 0 0 0 
team - not attended 0 0 0 team - not attended 0 0 0 
administrator - full 16 14 19 administrator — full 16 14 19 
administrator - partial 0 0 0 administrator — partial 0 0 0 
administrator - not attended 3 4 0 administrator - not attended 3 4 0 
Total number of teams 19 18 19 Total number of teams 19 18 19 


Overall satisfaction with training 


School teams were asked to indicate how much they agree or disagree that the training was a positive, worthwhile 
experience overall. The vast majority of school teams indicated satisfaction with the training experience. However, 
one or two schools the Metro region from both cohorts and the South region from Cohort 11 indicated some 


dissatisfaction. See Figure 2. 


2. School team ratings: “Overall, this training was a positive, worthwhile learning experience.” 


Cohort 10 Cohort 11 
Metro North South Metro North South 
Strongly agree 17% 25% 14% 40% 69% 29% 
Agree 712% 75% 86% 50% 31% 64% 
Disagree 6% 0% 0% 10% 0% 71% 
Strongly disagree 6% 0% 0% 0% 0% 0% 


Note: Combined percentages of the “strongly agree” and “agree” categories that equal 85% and above are highlighted in green indicating a 
high level of satisfaction. Combined percentages of the “disagree” and “strongly disagree” categories that equal 15% and above are highlighted 
in orange to indicate a relatively lower level of satisfaction (none on this table). 


Ratings of specific training content areas 


Cohort 10 


School teams were asked to rate how useful the specific training content areas are in terms of helping them 
implement PBIS at their school. For Cohort 10 (Year 2), “Action planning,” “Update TIC,” and “Sustaining PBIS,” 
are the sessions that were most likely to be rated as “vitally useful” or “very useful” for school teams in all 
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regions. While most school teams found each of the training sessions to be useful, there were some sessions that 
were not rated as highly. One-third of school teams from the Metro region found “Networking” to be “not at all 
useful.” One-fifth of school teams from the South region reported that the “Transition from TIC and TFI/Update Action 
Plans” session was “not at all useful.’ One-tenth of school teams from the north region thought the “Update TIC” 
session was “not at all useful.” See Figure 3. 


3. _ Ratings of specific training content areas — Cohort 10 (Year 2) 


Critical Features of Tier 3 
Slide and Planning for Tier 3 
(FBA and FBA World Café) 


Transition from TIC and 
TFl/Update Action Plans 


Team interviews for 
graduation ceremony 


TFI presentation and 
introduce TFI to the whole 
team 
Sustaining PBIS/End of 
Year/Beginning of Year 
Planning and Buy-in 


0% 20% 40% 60% 80% 100% 
= Vitally useful m= Very useful Somewhat useful mNotatalluseful ©N/A 


Additionally, schools from Cohort 10 were asked in an open-ended format what parts of training were the most and 
least useful to their PBIS teams. The overwhelming majority of school teams in all regions (n=27) stated that 
planning and work time were the most useful parts of their training. The networking session was particularly useful 
to schools in the South region (n=7). A few teams (n=6) from all regions found the FBA session to be helpful. The 
same number of schools from the Metro and North region thought the Sustaining PBIS session was helpful. Below 
are a few of their comments, edited for clarity: 


“The time we spent together as a team; the opportunity to work together and complete work and action planning.” 
—Cohort 10 team, North region 


“Networking and getting ideas from other schools. Talking about strategies to sustain our PBIS program as we move 
forward.’—Cohort 10 team, South region 
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“Sustaining PBIS. We are still working on getting whole-staff buy-in and modifying things from Year 1. We want to find ways 
to continue and build PBIS into all areas of our building and create a supportive environment for all members of our staff.” 
—Cohort 10 team, North region 


School teams from Cohort 10 also provided feedback on the least useful parts of the training. Teams had varying 
responses; however, a few school teams (n=10) from all regions thought the TFI session was the least useful. Eight 
school teams across all regions thought the FBA was the least useful part of training. Below are a few of their 
comments, edited for clarity: 


“The TF session—it was too rushed.’—Cohort 10 team, Metro region 
‘The TFI [session] had lots of jargon and it was difficult for staff sometimes.” —Cohort 10 team, North region 


‘FBA World Café—there was no new learning.” —Cohort 10 team, Metro region 


Cohort 11 


For Cohort 11 (Year 1), the “Action Planning,” “Steps & Products Checklist,” “Sustaining,” and “Chapter 4 (North 
Region only)’ sessions were most likely to be rated as “vitally useful” or “very useful” by teams in all regions. 
School teams found the sessions on “Networking,” “Poster board,” “TFI,” and “Step 5” to be “very” or “somewhat 
useful.” Very few school teams from the Metro (5%) and North (8%) regions reported that taking the TFI was 
“not at all useful.” See Figure 4 on the following page. 


Cohort 11 schools were also asked in an open-ended question format to identify the most and least useful training 
components. Similar to Cohort 10, the vast majority of school teams in Cohort 11 commented that the time to work 
together in teams was the most useful part of training. A few schools from the North and South regions (n=7) 
thought that the networking sessions were the most useful. Some schools from the Metro and South regions found 
the planning for the end of this year and beginning of next year to be useful. See below for a few of their 
comments, edited for clarity: 
“Having work time is the most useful for our team. We are very productive with our time. We also appreciate the perspective 
of where we are compared to other teams in the cohort.”—Cohort 11 Team, Metro region 
‘The most useful part for our team was the action planning, seeing what other schools are doing, and looking at all of the poster 
boards.” —Cohort 11 Team, North region 


‘It was good for us to plan for the end of the year and next year. Taking the TFI and working on the action plan provided specific 
tasks and projects for us to work on.’"—Cohort 11 Team, South region 


Additionally, Cohort 11 school teams provided feedback on what they found to be the least useful training 
components. Answers varied greatly across teams and regions. A few schools from the Metro and North region felt 
there was too much work time. Some schools from the North region thought the networking on data systems was not 
useful. Some schools from the North and South regions thought that there was repetitive information from previous 
trainings. See below for some of their comments: 


“The team work time was overall too lengthy. We felt the information in the two days could be [condensed] into one day.’—Cohort 
11 team, Metro region 


“SWIS Networking on Data Systems—we didn’t learn anything new since only one other school is using [another system] and their 
demographics were not like ours.’"—Cohort 11 team, North region 


‘A lot of information presented today and yesterday was repeated from previous training sessions. A lot of the information could 
have easily been pared down.’—Cohort 11 team, South region 
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4. Ratings of specific training content areas — Cohort 11 (Year 1) 


Chapter 4: classroom 
management practices and 
systems 


Special networking activity 
(North: Networking on data 
systems, South: Commitment 


Team sharing and rewards 
(North region only) 


Using outcome data: 
precision problem solving 


Sustaining: planning for the 
end of the year/beginning of 
next year 


Steps & products checklist 


Action Planning 


Step 5: PBIS classroom 
management practices and 
systems 


Posterboard 
(South region only) 


Networking 
(South region only) 


Networking and activities 
(posterboard) 


0% 20% 40% 60% 80% 100% 


= Vitally useful m Very useful Somewhat useful mNotAtAll Useful © N/A 
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Ratings of training effectiveness 


School teams were also asked how strongly they agree or disagree with five statements about the effectiveness of 
the trainings. A majority of all school teams in all regions reported that the training information was clearly 
presented, that the training will improve practices in their school, and that their team is confident to meet with 
school staff and share the topics they learned at the training. Some school teams from Cohort 11 in the South 
region disagreed the training enhanced their understanding of PBIS. A few teams from the Metro region in Cohort 
10 disagreed that the resources, materials, and teaching aids provided in training were helpful. See Figure 5. 


5. Ratings of training effectiveness 


Cohort 10 Cohort 11 

Metro North South Metro North South 
The training information was clearly presented. 
Strongly agree 12% 25% 14% 20% 69% 29% 
Agree 88% 75% 71% 80% 31% 64% 
Disagree 0% 0% 14% 0% 0% 7% 
Strongly disagree 0% 0% 0% 0% 0% 0% 
The training enhanced our understanding of PBIS. 
Strongly agree 6% 8% 7% 25% 69% 21% 
Agree 88% 83% 79% 65% 23% 50% 
Disagree 6% 8% 14% 10% 8% 29% 
Strongly disagree 0% 0% 0% 0% 0% 0% 
The training will improve our practices in school. 
Strongly agree 6% 33% 7% 35% 77% 29% 
Agree 88% 58% 93% 65% 23% 71% 
Disagree 6% 8% 0% 0% 0% 0% 
Strongly disagree 0% 0% 0% 0% 0% 0% 
The resources, materials, and teaching aids provided in training were helpful. 
Strongly agree 18% 17% 21% 25% 69% 29% 
Agree 65% 75% 79% 70% 31% 57% 
Disagree 18% 8% 0% 5% 0% 14% 
Strongly disagree 0% 0% 0% 0% 0% 0% 
Our team is confident in our ability to meet with school staff and share topics learned today. 
Strongly agree 29% 42% 29% 40% 38% 50% 
Agree 71% 50% 71% 55% 62% 50% 
Disagree 0% 8% 0% 5% 0% 0% 
Strongly disagree 0% 0% 0% 0% 0% 0% 


Note: Combined percentages of the “strongly agree” and “agree” categories that equal 85% and above are highlighted in green indicating 
a high level of satisfaction. Combined percentages of the “disagree” and “strongly disagree” categories that equal 15% and above are 


highlighted in orange to indicate a relatively lower level of satisfaction. 
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Completion of PBIS activities 


School teams were asked if they had completed key PBIS activities at training, or within the past month. A majority 
of school teams in all regions from both cohorts “fully completed” or “partially completed” their action plan, but 
29 percent of school teams in Cohort 10 from the South region have not completed it. A majority of school teams in 
all regions for Cohort 10 and Cohort 11 report that they have “fully completed” a fidelity assessment such as the TIC, 
SAS, SET, or TFI. School teams from all regions in Cohort 10 were more likely to report that they fully or partially 
reviewed their ODR data in the past month. A few school teams in all regions from Cohort 11 and the South region from 
Cohort 10 report that they did not review their ODR data in the past month. Use of data for planning and making 
decisions is a key element of training, so it would be expected that schools completing their second year should use 
data as a regular practice and are implementing this concept, whereas schools in their first year of training might 
just be establishing data systems and are just beginning to use ODR data. See Figure 6. 


6. Completion of PBIS activities at training or within the past month 


a Cohort 10 Cohort 11 
Activity 
Metro North South Metro North South 
Update your school’s PBIS Action plan 
Fully completed 56% 67% 14% 15% 38% 57% 
Partially completed 44% 33% 57% | 25% 62% 43% 


Not completed 0% 0% 29% | 0% 0% 0% 


Review your school’s Office Discipline Referral (ODR) Big 5 data 


Fully completed 63% 83% 21% 45% 31% 43% 
Partially completed 31% 8% 64% 45% 46% 43% 
Not completed 6% 3%, 

Taken a fidelity assessment (TIC, SAS, SET, TF, etc.) 
Fully completed 94% 83% 50% 80% 92% 93% 
Partially completed 6% 8% 43% 20% 8% 7% 
Not completed 0% 8% 7% 0% 0% 0% 


Note: Cells in the ‘fully completed” category that are 80% and above are highlighted in green to indicate a high level of activity completion. 
Cells in the “partially completed” category that are 50% and above are highlighted in blue to indicate a high level of partial completion. 
Cells in the “not completed” category that are 10% and above are highlighted in orange to indicate a low level of activity completion. 


Strengths and challenges implementing PBIS 


Schools from both cohorts in all regions were asked in an open-ended format to identify what successes and 
challenges they are facing with PBIS implementation. 


Successes 


When asked about their biggest successes in implementing PBIS, Cohort 10 school teams across all regions commonly 
identified rewards systems (n=27) and defining expectations (n=8). Schools from the Metro and South regions (n=7) 
also listed buy-in as a strength of their PBIS initiative. Below are some of their comments, edited for clarity: 


“Our reward system and coming up with five school-wide expectations.’—Cohort 10 team, North region 
“Student recognition with PBIS—we have weekly drawings, monthly celebrations, etc.’--Cohort 10 team, South region 
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School teams in Cohort 11 from all regions identified establishing a rewards system (n=25) and buy-in (n=7) as 
the most successful components of their PBIS program. A few teams in the Metro and South regions identified the 
Teaching PBIS Expectations (n=5) as a success. A smaller number (n=4) teams from the South and North regions 
indicated they have had success with focusing on specific locations in their school. A few (n=3) teams from the South 
region said that that their team was a successful part of their PBIS program. Below are some of their comments, 
edited for clarity: 


“[Our] Pride points and rewards system [has been a success]. We have full support from our school and are fully funding 
our rewards.’—Cohort 11 team, North region 


“Our agreement on school-wide expectations and implementation of school-wide lessons [has been a success].’—Cohort 
11 team, Metro region 


‘In our cafeteria, we made some changes and they have paid off with behavior and time management.’—Cohort 11 team, 
South region 


“[Our] team and staff investment [are strengths of our PBIS program].’—Cohort 11 team, South region 


Challenges 


Cohort 10 schools in all regions identified buy-in (n=11), consistency (n=9), and staff issues (such as new staff and 
staff turnover) (n=5) as the most common challenges their teams face with PBIS implementation. The North and 
Metro regions identified using data (n=6) as a challenge, and the Metro and South regions (n=6) noted that time as 
a challenge for PBIS implementation in their schools. Below are some of their comments, edited for clarity: 


“The staff who don't buy-in really don’t like the program.’—Cohort 10 team, Metro region 
“Having enough time to meet as a team [is a challenge].’—Cohort 10 team, South region 


Half of our staff is new this year. Trying to get new staff, including an administrator, trained on the matrix, positive behavior 
Slips, and general continuity of behavior expectations throughout the day and the building [is a challenge].” 
—Cohort 10 team, Metro region 


“Staff buy-in and consistency with response to behavior [are challenges for our school].’"—Cohort 10 team, North region 


Many Cohort 11 schools across all regions identified the biggest challenges in implementing PBIS at their school to 
be staff buy-in (n=22), using data (n=11) and time (n=10). See below for a few of their comments, edited for clarity: 


‘We struggle with] time to train non-certified staff and understanding how to collect our data, how to get in an excel format, 
how to get our teachers to enter correctly. —Cohort 11 team, North region 


“We struggle with] staff buy-in and finding and scheduling a time to update all staff [about PBIS]: teachers, support staff, 
Janitors, etc.’—Cohort 11 team, North region 


“(We struggle with] keeping everyone on board and positive.’-—Cohort 11 team, South region 


Implementation fidelity measures 


PBIS is an evidence-based approach to addressing behavior issues in schools. A significant amount of research has 
been done to identify the critical features of PBIS. More generally, implementation science points to a specific 
sequence to ensure that PBIS is implemented with fidelity. The following assessment tools are used to assess PBIS 
implementation fidelity among participating schools in Minnesota: Team Implementation Checklist (TIC), Self- 
Assessment Survey (SAS), School-wide Evaluation Tool (SET), Tiered Fidelity Inventory (TFI) (Cohort 11 only), 
and the Benchmarks of Quality (BoQ). The BoQ is not used with schools that are in training. The TFI is a new 
instrument that measures implementation fidelity on all three tiers of PBIS and effectively replaces the TIC and the SET. 
It was rolled out for use in Minnesota schools in spring 2015. Currently, only Cohort 11 is using this assessment. Cohort 
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10 will still utilize the TIC and the SET to assess their implementation fidelity, but will be given the option to use 
the TFI once they have completed training. The number of TICs and SASs that were completed, and TFI scores 
from August 2015-March 2016 are reported here as well the degree to which school teams completed PBIS 
activities that contribute to implementation fidelity. 


PBIS Activities 


School teams are asked to report in the training survey the extent to which they completed the following PBIS 
activities either at training or in the past few weeks: updating their PBIS action plan, reviewing ODR data, and 
whether or not they have taken a fidelity assessment. Overall, the majority of school teams in all regions from 
both cohorts reported that they either have “fully completed” or “partially completed” all three PBIS activities. 
Schools from Cohort 10 were more likely to report that they had “not completed” any of the three activities. Some 
school teams from both cohorts report they had “not completed” a review of their ODR data. See Figure 7. 


7. Completion of PBIS Activities by cohort (Spring 2016) 


Cohort 10 (n=41) Cohort 11 (n=53) 
10% 10% 5% 
ae a — 15%| 
20% | o | a 
40%} 
42% 
Update Action Review ODR Take Fidelity Update Action Review ODR Take Fidelity 
Plan Data Assessment Plan Data Assessment 
«Fully =Partially =Not completed = Fully =Partially = Not completed 


Team Implementation Checklist 


The Team Implementation Checklist, or TIC, is a tool used by schools to measure PBIS implementation fidelity. 
The TIC is designed to be completed by the PBIS Team three times per year to monitor activities for 
implementation of PBIS in a school. In order to be on track with MDE’s data calendar, Cohort 10 schools in training 
should complete at least one TIC by the end of August and another one by the end of November. School teams are 
instructed to complete the TIC at the Fall, Winter, and Spring PBIS training sessions. Most schools have 
completed the TIC three times, however, there are a few schools that have completed two TICs. The number of 
TICs completed by Cohort 10 as of May 2016 is included in Figure 8 below. 


8. Number of TICs completed by spring 2016 (by region) 


Cohort 10 (N=52 0 TICs 1TIC 2 TICs 3 TICs 
Metro (N=21) 0 0 3 18 
North (N=18) 0 0 2 16 
South (N=13) 0 0 3 10 
Total for cohort 10 0 0 8 44 
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Strengths and challenges of the TIC 


School teams in Cohort 10 completed the Team Implementation Checklist (TIC) at training and were asked to 
identify areas of strengths and challenges based on their TIC outcomes. Most teams in all regions reported 
“Establish commitment,” “Establish and maintain team,” “Self-assessment,” “Classroom behavior support 
systems,” and “Establish prevention systems” to be a strength. Most school teams in the Metro region report 
“Establish information system” to be a strength while half of school teams from the South region reported this to 
be “a challenge.” Two-fifths of school teams in the North region reported “Establish information system” to be “a 
challenge.” Half of school teams from the Metro and a few more teams in the South find “Building Capacity for 
Function-Based Support” to be a challenge. A few schools from the North and South regions state that they “have 
not started” this component of PBIS. See Figure 9. 


9. Strengths and challenges of the TIC—Cohort 10 (Spring 2016) 


Cohort 10 
Activity Metro North South 
Establish commitment 
It's a strength 100% 67% ™M% 
It's a challenge 0% 33% 29% 
Have not started 0% 0% 0% 
Establish and maintain team 
It's a strength 94% 83% 64% 
It's a challenge 6% 17% 36% 
Have not started 0% 0% 0% 
Self-assessment 
It's a strength 88% 67% 57% 
It's a challenge 12% 33% 36% 
Have not started 0% 0% 1% 
Establish school-wide expectations: prevention systems 
It's a strength 15% 100% 86% 
It's a challenge 25% 0% 14% 
Have not started 0% 0% 0% 
Classroom behavior support systems 
It's a strength 63% 58% 64% 
It's a challenge 38% 42% 36% 
Have not started 0% 0% 0% 
Cohort 10 
Activity Metro North South 
Establish information system 
It's a strength 81% 58% 50% 
It's a challenge 19% 42% 50% 
Have not started 0% 0% 0% 
Build capacity for function-based support 
It's a strength 50% 58% 36% 
It's a challenge 50% 25% 57% 


Have not started 0% 16% 1% 


Note: Cells in the “it’s a strength” category that are 80% and above are highlighted in green to indicate an area of strength. Cells in the “it’s 
a challenge” category that are 50% and above are highlighted in blue to indicate an area of challenge. Cells in the ‘not completed” 
category that are 10% and above are highlighted in orange to indicate a low level of activity completion. 
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Tiered Fidelity Inventory 


The Tiered Fidelity Inventory (TF) examines how schools are implementing across all three tiers of PBIS. The 
TFI is designed to be taken four times per year while the school is in training. Schools will take a TFI at each 
PBIS training and will have one TFI assessment completed at their school with an external evaluator (who is a 
member of another school’s PBIS team matched to this school using a “no-cost exchange” model). The external 
evaluator will visit their school and complete a “walk-through” to interview staff and students in addition to 
completing the coaching session component. This year, only Cohort 11 schools completed the TFI at each of the 
three trainings. Note that Cohort 11 schools only completed the Tier | portion of the TFI assessment. 


In order to implement PBIS with fidelity, a school needs to score 70 percent or better on their TFI for each of the 
three tiers of PBIS. Fifty-five school teams completed a TFI at each of the trainings this year. Average TFI scores 
increased for each region: the average score for all schools in each region for Tier | in the Fall was 22 percent; the 
average increased to 39 percent in the Winter, and increased again to 57 percent in the Spring. See Figure 10 for 
average TFI scores for Tier 1 separated by region. 


10. Average Tier 1 TFI scores by region—Cohort 11 (2015-2016 school year) 


60% 

(n=18) 
43% 46% 
(n=18) 


56% 49 57% 
(n=18) Rati (n=55) 


Metro North South Total average across 
all regions for 15-16 
school year 


= Fall 2015 = Winter 2015 = Spring 2016 


School teams in Cohort 11 that completed the TFI at spring training and were asked to identify areas of strengths 
and challenges based on their TFI evaluation outcomes. Nearly all school teams from all regions reported that the 
“Teams” component to be a strength. Most school teams from all regions report that “Implementation” and 
“Evaluation” to be a challenge. Some school teams from the North region and a greater number of teams from the 
South region note that they have not started the “Evaluation” component of their PBIS program. See Figure 11. 
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11. Strengths and challenges of the TFI—Cohort 11 


Cohort 11 
Activity Metro North South 
Teams 
It's a strength 95% 85% 100% 
It's a challenge 5% 15% 0% 
Have not started 0% 0% 0% 
Implementation 
It's a strength 25% 15% 21% 
It's a challenge 75% 17% 19% 
Evaluation 
It's a strength 20% 23% 21% 
It's a challenge 80% 54% 50% 
Have not started 0% 


Note: Cells in the ‘it’s a strength” category that are 80% and above are highlighted in green to indicate an area of strength. Cells in the “it's a 
challenge” category that are 50% and above are highlighted in blue to indicate an area of challenge. Cells in the “not completed” category that 
are 10% and above are highlighted in orange to indicate a low level of activity completion. 


Self-Assessment Survey 

The Self-Assessment Survey (SAS) examines the status and need for improvement of four behavior support 
systems: school-wide, non-classroom, classroom, and individual students. A summary of the survey results is 
used to develop an action plan for implementing and sustaining PBIS systems throughout the school. 

For Cohort 11 schools, the SAS should be completed once in the fall and once in the spring to be on track with 
MDE’s data calendar. Many schools in Cohort 11 have completed the SAS at least once this school year (n=45); 


however, 11 schools have not yet completed a SAS this school year from Cohort 11. 


Eighteen schools from Cohort 10 have completed the SAS at least once this year—the SAS only needs to be 
completed once during the school year for schools in the second year of PBIS training. See Figure 12. 


12. Number of SAS completed as of spring 2016 (by region) 


Cohort 10 (N=52) 0 SAS 1 SAS 2+ SAS 
Metro (N=21) 11 9 1 
North (N=18) 12 6 0 
South (N=13) 6 3 4 
Total for cohort 10 29 18 5 
Cohort 11 (N=56) 0 SAS 1 SAS 2+ SAS 
Metro (N=19) 3 13 3 
North (N=18) 2 11 5 
South (N=19) 6 10 3 
Total for cohort 11 11 34 11 
Total for cohorts 10 & 11 combined (N=108) 40 52 16 
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Behavioral data system 


The presence of a behavioral data system allows for schools to track office discipline referral (ODR) data and 

use this for data-based decision-making. The data collected in these systems are used for improving school-wide 
behavior support. Schools should have an established behavior system in place by the time of Winter training in 
their first year. More schools in Cohort 11 than Cohort 10 use SWIS. The same number of schools in Cohorts 10 
and 11 use a data system other than SWIS. One school from each cohort is identified as having SWIS and another 
data system listed to track student behavior. See Figure 13. 


13. Behavioral data system used, Cohorts 10 & 11 


38 
34 


Cohort 10 (N=52) Cohort 11 (N=56) 


= SWIS only 
= SWIS and some other data system 
= Non-SWIS 

Unknown/None 


Issues to consider and recommendations 


Based on the results described above, there are a few things the Minnesota PBIS Statewide Leadership Team, including 
the Minnesota Department of Education (MDE) and the Regional Implementation Partners (RIPs) should consider 
when planning future PBIS trainings. RIPs should also pay attention to areas where their particular region or a 
specific Cohort within their region may be rating aspects of the training less favorably, or where their school’s 
implementation data may be less positive, to determine possible causes and solutions for these specific problem areas. 


Use of data and action planning. Most schools in Cohort 10 and 11 completed the TIC or TFI at training or within 
the weeks prior to training. Most Cohort 10 school teams fully or partially updated their school’s PBIS Action 
Plan. A majority of Cohort 11 schools partially updated their Action Plan. The majority of Cohort 10 and 11 teams 
either fully or partially reviewed their school’s ODR data. However, there were some schools from all regions in 
each cohort that reported this activity as “not completed.” Furthermore, using and collecting data is mentioned 
frequently by school teams from all regions in Cohort 11 and the Metro and North regions in Cohort 10 as one of 
the biggest challenges they have to implementing PBIS. 


While trainers and RIPs should continue to emphasize how important using data is to PBIS implementation and 
sustainability, it might be beneficial to hold a mini data analysis workshop (similar to MRIP Data Day) at one 
training per year so schools have the opportunity to dive into their ODR and implementation fidelity data with 
trainers present. This may also give schools who feel that there is “too much team work time” some direction in 
their action planning time. It could also be an opportunity for schools to look at ODR data in different ways. 
Looking at data at training as part of a mini work shop also ensures that their data is current and useable for Action 
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Planning. We also recommend that MDE and the RIPs continue to ensure time is set aside at training and that 
technical assistance, specifically centered around schools completing their Action Plan, looking at data, and setting 
up time to take fidelity assessments, is provided to schools throughout the two-year training sequence to review 
their outcome data. As one Cohort 10 school from the South region put it: 


‘Please include time for each district/school to bring their office discipline referral data with them, analyze, and action 
plan.’—Cohort 10 team, South region 


TFI. Some (n=10) school teams from Cohort 10 found the session about the TFI to be the least useful part of 
training. The trainers and RIPs should continue to emphasize the importance of the TFI as a tool that can be used to 
measure implementation fidelity and the benefits of completing this assessment (1.e., it replaces the TIC and the 
SET, is useful for Action Planning, etc.) This summer, Wilder Research is going to create a webinar/training video 
and additional supplemental materials about the TFI to help schools better understand what the tool is, how to 
conduct a TFI, and how it is useful for school teams. These tools will be available for all schools to use on the 
Minnesota PBIS website. Wilder Research will continue to attend PBIS training to assist with TFI matching and 
training coaches and new TFI facilitators. As trainers and school teams become more familiar with the TFI, it is 
likely that there will be more ideas for how to make the TFI the most useful. The RIPs and MDE should find a way 
to document these ideas and implement them more systematically in the upcoming school years. 


Buy-in. Staff and student buy-in is a major area that needs to be addressed in training. When asked in the open- 
ended format, some Cohort 10 (n=11) and Cohort 11 (n=22) school teams in all regions identified buy-in as one of 
the biggest challenges to implementing PBIS at their school. The RIPs/trainers should consider ongoing training 
for obtaining and sustaining staff and student buy-in and providing concrete examples of how to do this at each 
training session. 


Training format, content, and attendance. Overall, the training was well-attended by school teams and administrators. In 
fact, all school teams in both cohorts had perfect attendance for Spring training. However, some administrators in the 
Metro and North regions in both cohorts did not attend training. The RIPs should continue to emphasize the importance 
of having an administrator present at the training and involved in their school teams. If they have not already done so, 
the SLT and the RIPs should consider following up with administrators to find out barriers to their attendance. Or, 
alternative training formats for administrators could be offered (recorded trainings, webinars, etc.) if they are unable to 
attend PBIS training. Some school teams from the South region in Cohorts 10 and 11 (n=4) suggested having a virtual 
training so it is more accessible to attend: 


‘We would like to have] a webinar on the second day instead of onsite training. We could do a lot of team building and 
planning at our school and save on drive time.’—Cohort 11 team, South region 


Additionally, a few schools from the Metro region in both Cohorts offered a variety of suggestions to improve 
training format: 
“Could the lesson information set up as optional stations? That way school teams who want to know more about a certain 


topic could choose to attend a lesson and school who feel confident and need more time could continue to work?’—Cohort 
11 team, Metro region 


“More secondary-specific examples are needed—maybe split into two cohorts for elementary and secondary?’—Cohort 11 
team, Metro region 


“| think [it would be great] if there is a "catch-up" program we could send new staff to so we arent training them while trying 
to learn the program.’—Cohort 10 team, Metro region 
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One other option would be to consider incorporating into the scope and sequence of training another two or more 
half-day “in school” retreats for the PBIS team. The teams would stay at their school and conduct action planning, 
examine ODR and fidelity data, etc. and possibly partner with an outside RIP or trainer to be present to support 
their working time. 


Examples and materials. The majority of teams in all regions in Cohort 10 and the North region in Cohort 11 believed 
the resources, materials, and teaching aids were helpful. However, a few school teams from the Metro and North 
regions in Cohort 10 and the Metro and South regions from Cohort 11 disagreed that these materials were helpful. 
MDE and the RIPs should consider ways of improving resources, activities, and teaching aides that are provided 
during training. Many school teams from each cohort noted the need for more concrete, specific examples of how 
to implement PBIS. School teams in each cohort provided a variety of answers in the open-ended format 
regarding specific examples they would like to see at PBIS training: 


“Make materials more accessible. Give specific examples for communicating to our staff (step by step examples for how a 
school shared their data with their staff), more opportunities to interact with other buildings to get ideas on what they are 
doing.’—Cohort 11 team, Metro region 


“Could we see some additional strategies for our Tier II and III students?’—Cohort 11 team, Metro region 
“We can always use more information on positive behavior strategies and interventions.’—Cohort 11 team, South region 


‘[There has] to be more high school examples. This is much easier [to implement] at a younger level, and we would like to 
see more concrete [examples of PBIS at] high school levels."—Cohort 11 team, South region 


“We would like more examples of Tier II and III interventions and exemplary schools.’—Cohort 10 team, Metro region 


“Could we see more examples from Level-lV schools?’—Cohort 11 team, North region 


New training survey and individual trainer feedback. This school year, Wilder Research attempted to collect data on 
individual trainers to try to make the survey tailored to regions and their training topics and to inform trainers of 
any areas they might want to change for their trainings. Results from these questions were not reported, but were 
made available to the SLT and the RIPs as well as the PBIS trainers. 


Wilder Research and the SLT have formed a training survey work group to re-design the current training survey 
to make it more responsive to evolving training format. The new training survey will ask about what teams do 
during their action planning time, knowledge gained at training, implementation fidelity activities completed, 
what parts of training were the most useful, and more. In addition to designing the new training survey, Wilder 
Research will also create questions for individual trainers to use to gather feedback (if they choose) about their 
specific sessions. These survey questions will not be included in the main training survey, and it will be the 
trainer’s responsibility to distribute the training surveys and compile the results. This decision came from the 
work group, which believed that feedback about specific trainers should be kept separate from the PBIS training 
survey. We hope this separate survey will produce meaningful feedback for trainers. 
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For more information 
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